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Introduction 


Automation related accidents or serious incidents are not limited to advanced technology 
aircraft. There is a full history of such accidents with conventional technology aircraft. 
However, this type of occurrence is far from sparing the newest “glass cockpit” generation, and 
it even seems to be a growing contributor to its accident rate. Nevertheless, all these aircraft 
have been properly certificated according to the relevant airworthiness regulations. Therefore, 
there is a growing concern that with the technological advancement of air transport aircraft 
cockpits, the current airworthiness regulations addressing cockpit design and human factors 
may have reached some level of inadequacy. This paper reviews some aspects of the current 
airworthiness regulations and certification process related to human factors of cockpit design 
and focuses on questioning their ability to guarantee the intended safety objectives. 


Current Certification Principles 


Certification Purposes and References 

According to Article 31 of the Convention on International Civil Aviation (Chicago, 1944), any 
aircraft involved in international operation shall hold a certificate of airworthiness delivered or 
validated by its state of registry. This is intended to achieve protection of other aircraft, third 
persons and ground property while an aircraft registered in state B flies into or over the territory 
of state A. Following article 33 of the Convention, Annex 8 to the Convention includes broad 
standards which define the minimum international basis for the recognition by states of 
certificates of airworthiness delivered or validated by other states. Furthermore, Annex 8 sets 
the minimum international degree of standardization called upon by article 37 of the 
Convention. 
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However, as a matter of fact, the main purpose of airworthiness certification has been the 
protection of the passengers and crewmembers for a long time. This is a much more demanding 
objective, which has been achieved trough the implementation of national codes of 
airworthiness containing the full scope of requirements considered necessary by the states to 
reach the target safety level. 

Today, only two airworthiness codes form the potential reference for any transport category 
aircraft certification: the U.S. FAR 25, and the European JAR 25. Due to the pressure of a 
highly internationalized business, these two codes are only differentiated by minor differences, 
and can therefore be referred to as a single reference for the present discussion. 


Basic Principles 

The airworthiness requirements concerning the cockpit, as for any other subsystem of the 
aircraft, are not aiming at any “best possible” design, but they intend to specify the minimum 
objectives to be matched by an applicant design. This is a very basic principle of any 
certification. 

As far as human factors in cockpit design and equipment are concerned, the minimum 
objectives currently set by the airworthiness code are more or less limited to the following: 

• To guarantee that the minimum crew (i.e., after one crew member incapacitation) is 
still able to do the job without excessive workload or fatigue (FAR/JAR 25-1523) 

• To provide the crewmembers with acceptable comfort and protection against outside 
conditions, so that they can do their job without excessive effort, concentration or 
fatigue (FAR/JAR 25-77 1 ) 

• To provide the crew with a sufficient visibility to the outside (FAR/JAR 25-7) 

• To minimize the risks of mistake in the controls use, particularly through a 
standardization of the shape and movements of the primary flight controls (FAR/JAR 
25-777; 781) 

• To minimize ambiguities in the information displayed by the instruments (FAR/JAR 
25-1303; 1321; 1322) 

• To provide the crew with relevant alerting information about unsafe functioning states 
of any equipment or system, and to allow appropriate crew action (FAR/JAR 25- 
1309). 

However, these generic requirements are completed by a set of “special conditions,” adapted 
to the specifics of each particular aircraft. These special conditions may well include extensive 
and detailed requirements for systems like CRT display flight instruments. 


Demonstration of Compliance 

The methodology used to check the compliance of a proposed design with a relevant 
airworthiness requirement heavily depends on the explicit versus implicit nature of the 
requirement. 
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Explicit requirements are directly expressed in terms of design characteristics. For example, 
FAR/JAR 25-781 quotes: “cockpit control knobs must conform to the general shape... in the 
following figures.” In this case, the compliance of a proposed design is rather easy to check, 
and direct examination of descriptive material (drawings, scale models, mock-ups) can be used. 

However, most of the human factors related issues are covered by implicit requirements, 
expressed in terms of general outcomes to be achieved. For example, FAR/JAR 25- 111 (a) 
quotes: “each cockpit control must be located to provide convenient operation and to prevent 
confusion and inadvertent operation”. 

In the later case, the methodology used to evaluate the ability of a proposed design to reach 
the objective obviously is the critical part of the certification process. A first possible source of 
difficulty is the interpretation of the regulatory objective itself. A second possible source of 
difficulty is the acceptable means of compliance with the (interpreted) objective. Consequently 
some regulatory requirements are complemented with advisory material, including interpretation 
guidelines and/or indications on acceptable means of compliance. Acceptable means of 
compliance more often than not are proven solutions, or sets of solutions, that have been 
shown by service history to be satisfactory. 


The Test Pilot Judgment Methodology 

As a matter of fact, one of the only tools currently in use to evaluate a new design human 
factors acceptability in a certification process is test pilot judgment . This judgment is based on 
regulatory and company test pilots comparative experience, following from actual or simulated 
flight exposure (several thousands of hours for major test programs) in the subject cockpit on 
the one hand, and previous experience with existing designs on the other hand. In other words, 
this judgment is based on extrapolations to the new design of expertise gained on the previous 
ones. 

Furthermore, the certification process cannot wait until the first aircraft prototype is built to 
start up. No manufacturer would take the risk of becoming involved in such highly expensive 
development programs without reasonable guarantee that the projected designs are certifiable. 
Consequently new designs are submitted to regulatory authorities to get some “certifiability” 
agreement well before a prototype aircraft is built. In this situation, pilot judgment cannot be 
exercised in a real cockpit (real flight context) but has to be exercised in a mock-up or some 
other form of simulated environment. 

At the end of major certification programs, certification authorities are nowadays calling for 
operational route proving programs. These programs are the occasion for evaluating the aircraft 
in an airline-type environment, including “natural” and artificially induced failures, with mixed 
crews being composed of airline pilots and manufacturer test pilots. They have included up to 
one hundred flights in one occasion. 


Discussion 


The history of automation related accidents or serious incidents (see accident/incident reports 
list in annex) includes conventional generation aircraft such as the Lockeed L-101 1 Tristar, the 
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McDonnell Douglas DC- 10, and the Boeing B-747. This type of occurrence even seems to be a 
growing contributor to the newest “glass cockpit” generation accident rate. This may be an 
incentive to question the current human factors certification process of advanced technology 
aircraft. 

The protection ability of a certification process depends on several factors, including, but not 
limited to: 

• The adequacy of the requirements in expressing relevant safety objectives 

• The adequacy of the compliance checking methodology in use. 

As far as the human factors certification process of advanced technology aircraft is 
concerned, it seems that critics may be surprised by both of these aspects. 


Some Potential Biases in the Current Human Factors Certification Objectives 

Airworthiness codes and their objectives are not floating by themselves in the vacuum. They are 
embedded in a global aviation safety system, including components such as personnel 
certification (selection, training, proficiency checking), or operational procedures certification. 
The design and functional characteristics of this safety system reflect specific theories about risk 
and safety in the aviation transportation system. These theories are far from being mere rational 
constructions, consistent with all the available scientific evidences. They also are historical and 
social outcomes , conveying the current fears and faiths of the aviation community. 

The cutting up of the different safety codes and their role distribution is a first indicator of 
the background safety approach. This is particularly perceptible with the human error question. 
On the one hand, it is widely claimed that pilot error is contributing to about 70 percent of air 
transport accidents. On the other hand, merely four paragraphs (§ 777, 781, 1303, 1309) of the 
airworthiness code, out of 330, explicitly or implicitly address pilot error. This one percent 
order score suggests an implicit assumption that pilot error is neither really associated with 
airworthiness nor a design concern, but is much more related to pilot certification and 
procedures and operational regulations. As a matter of fact, this is the prevailing theory in the 
airworthiness world, and all the publications of the human factors researchers have had little 
influence on it up to now, except perhaps for this ironical one: a shift as occurred from 
culpability -based theories (pilot should not make errors) to fatalism (errors are normal and will 
always occur, whatever the design - so let's reinforce cross-check procedures... and substitute 
automation for pilot action as far as possible). 

A second illustration is offered by the workload assessment focus in the certification of the 
last ten years cockpit generation. Even with the same reference airworthiness code, large 
variations generally occur in the selection of items subject to specific attention in the certification 
process of different aircraft. Great departures from previous designs will normally be given a 
closer look. This is true for technical designs and for human factors aspects as well. The glass 
cockpit generation has progressively brought a drastic change to the previous pilot 
environment, including two-crew design, computer generated displays, sophisticated automated 
flight controls, and flight management computers. These changes presented a lot of challenges 
to pilots, such as autopilot active mode awareness, total energy awareness, crew 
communication, automation over-reliance, and computer interface problems. But in practice, 
one single question stands above everything else: the workload question. An ad hoc 
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Presidential Commission was set up in the United States in 1980 to endorse the concept. Since 
then, much effort, research and flight test time have been spent for evaluation, rating and 
judgment of workload levels during certification programs for the purpose of minimum crew 
complement assessment. 

At this point, it is rather difficult to elude some paradoxical feeling about the situation. On 
the one hand, great efforts are made about work load certification, but there is no real history of 
overload related accident on glass cockpit aircraft. This could be interpreted as the nice benefit 
of a particularly effective certification process on cockpit designs. Unfortunately, workload 
certification programs take place so late in the certification process that it is hard to imagine any 
significant change in the cockpit at this time, except for associated procedures. (It is not the 
intention here to minimize the importance of minimum crew evaluation, but only to suggest that 
its perceived importance is also subjective, crystallizing the socially highly sensitive skip to the 
two-crew cockpit.) 

On the other hand, there is a history of automation related accidents, but far less efforts are 
devoted to the certification of the related design aspects since they are not really felt to be 
airworthiness matters. 


The Adequacy of the Compliance Checking Methodology 

As already stated previously, nearly the only tool currently in use in a certification process to 
evaluate the acceptability of a new design in terms of human factors is test pilot judgment. This 
judgment is based on extrapolations to the proposed new design of expertise gained on the 
previous ones. Furthermore, as new designs are submitted to regulatory authorities to get some 
“certifiability” agreement well before a prototype aircraft is built, it often has to be exercised in 
simulated environment. 

This test pilot assessment methodology is by nature affected by some biases. 

First, it is based on the assumption that the experience of test pilots on previous aircraft is 
transferable on the new one. This may not be true for great developments in human-machine 
interface design. Secondly test pilots are not a representative sample of the airline pilots 
population. They have a very specific knowledge of the aircraft, which leads to different mental 
models of the aircraft. They do not share the daily routine operations and the associated 
constraints, and therefore the cognitive processes involved at the crew/aircraft interface are very 
doubtfully the same for test pilots and for airline pilots. Furthermore, typical figures for the 
number of individuals involved is about ten or a few tens and the total exposure time about two 
thousand hours. This is to be compared to the frequency of the hunted critical events or 
combinations of events, which is more likely to be of the order of one per hundred thousand 
hours. And to make things worse, recent studies (Amalberti & Wilbaux,1994) indicate that 
cognitive behaviors evolve a lot during the training process on glass cockpit aircraft, and that 
the average experience needed for the training process to reach a maturity stage and stabilize the 
cognitive behaviors is about 800 hours, or one and a half years. This is by far a figure that no 
test pilot will reach during a typical test period with a new type of aircraft. As a consequence, 
test pilot judgment will be exercised within a cognitive frame and the typical errors encountered 
in testing, which is significantly different from the average airline pilot situation. 

Finally, an individual judgment, even from a test pilot, would only provide rather soft 
grounds to refuse a proposed design. 
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Conclusion 


The current aircraft airworthiness certification requirements addressing human factors issues are 
expressed in rather general terms, and they are subject to interpretation uncertainties. This leads 
to an increasing inconsistency between the certification requirements and the new types of 
human-machine interface problems brought in by the glass cockpit highly automated aircraft 
generation. Consequently, it seems that there is a need for a redefinition of the objectives of the 
human factors certification process. 

The current evaluation process for the certification of human factors related aspects of 
cockpit design rely almost entirely on test pilot judgment This method has proven satisfactory 
for the past years, although it is marred by several biases. However, the changes induced by 
the new design of the pilot-aircraft interface has recently increased the effects of these biases to 
a significant degree. Consequently, it seems that there is a need for the development of human 
factors assessment protocols to complement test pilot assessment methodology. 
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